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ABSTRACT 

Implementation of recommendations of the Review of Management 
of the Department of the Navy and recent trends in Department of Defense 
policies require that the Bureau of Yards and Docks maintain a continuing 
effort to maximize the effectiveness of facilities maintenance resources. 
A means of measuring maintenance effectiveness would greatly assist in 
determining the capabilities of present systems and in spotting areas 
where greater improvement is needed. The concept of effectiveness is 
explained, and basic conceptual steps to be taken in its attainment 
outlined. Recent maintenance developments and operations in private 
industry and public agencies within the State of California are summarized. 
A proposal is made for use of military worth concepts in determining rela- 
tive priority of work requests at the local level, including a report of 
two introductory experiments to test the theoretical application recom- 
mended. An “ideal” measurement system is outlined, with major 
measurement problems discussed and recommendations for further study 


listed. 
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CHAPTER I 
BACKGROUND 

IMPLEMENTATION OF DILLON REPORT RECOMMENDATIONS. 

The Bureau of Yards and Docks has historically been a leader in attempts 
to improve the efficiency and effectiveness of facility maintenance pro- 
grams. It has long been recognized that budgets for maintenance of 
Naval facilities are severely limited, and that the utmost in actual 
effective maintenance effort must be realized from each maintenance 
dollar expended. 

Of particular importance in recent months, however, is the in= 
creased facilities maintenance responsibility assigned to the Bureau of 
Yards and Docks as a result of the approval of recommendations 76 and 77 
of the Review of the Management of the Department of the Navy = the 
"Dillon Report." Recommendation 76 of that report provides for assigning 
to the Chief of the Bureau of Yards and Docks the responsibility for main- 
tenance of buildings, grounds and structures and the operation of utilities, 
except for the Marine Corps a Recommendation 77 would assign to the 
Bureau of Yards and Docks the responsibility to devise and implement a 
management-oriented budget and information system for public works 
meregmeent .- The budget and information system would integrate present 
data-collection systems, improving them aS necessary to increase the 

United States Department of the Navy, Review of Management of 


the Navy, Vol. I (Washington: Department of the Navy, 15 December 1962), 
pe 27. 
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effectiveness of the budgeting and appraisal functions. The Secretary of 
the Navy has generally approved the recommendations of the Review of 
Management of the Department of the Navy and has specifically approved 
the concept of a single manager for maintenance of facilities. 
Implementation of these recommendations in the fullest sense will 
require the Bureau of Yards and Docks, through their Field Engineering 
Offices, to be able to review and analyze the maintenance activities of 
the Navy to obtain some indication of each activity's maintenance effective- 
ness. A successful reviewing tool would greatly assist the Bureau of Yards 
and Docks in knowing where best to apply the limited maintenance funds 
which will be available. Without a means of measuring maintenance 
effectiveness it would be difficult to avoid major errors in the allocation 
of maintenance resources. Such an evaluation tool must also be capable 
of working effectively across Management Bureau lines of interest as these 
may apply to each of the individual activities being evaluated. The system 
must also be responsive to the needs of the new Fleet Activities Command, 
which will influence the facilities management area in some, as yet not 
precisely defined, manner. The concept of a single=-manager for facili- 
ties maintenance is actually in conflict with the concept of a shore-based 
Fleet Activities Commander who is expected to exercise central control 


3 
secNav letter of 9 February 1963 to all ships and stations, 
subj: Report on "Review of Management of the Department of the Navy". 





over those elements in direct support of fleet forces 4 This conflict 
may present a major challenge to any system of measuring maintenance 
effectiveness. Any measurement tool used must be capable of satisfying 
extensive inter-agency and inter-office criticism of resource allocations. 

Department of Defense trends. Probably the kevnote of the current 
Department of Defense administration has been the increased use of 
economic analysis methods in consideration of major policy decisions. 
The TFX aircraft contractor selection and decision to use conventional 
propulsion in the Navy's next aircraft carrier are well known examples 
of cases where the results of the Department of Defense analysis were 
not favorable to the stated desires of the Navy. The Navy was not able 
to present its case so as to result in a favorable “cost effectiveness" 
analysis. 

The tendency of the Department of Defense to centralize common 
Support services under a single office, either within a single service, 
or within a new central Department of Defense agency, must also be 
noted. These changes are to be encouraged where savings in critical 
resources or improvements in operation are to be gained thereby, but 
must be resisted where the increased centralization will add little or 
nothing to improvement in the end product. In many instances, de- 
centralization of authority and control may be more effective in solution 

4 United States Department of the Navy, Review of Management of 
the Department of the Navy, Vol. II, Study 6 of 7 Studies, Facilities 


Management Study, Vol. I (Washington: Department of the Navy, 
26 October, 1962), p. 9 





of a problem than increased centralization. Decisions of this nature 
must be based on a thorough knowledge of the current situation, and 
the best possible rational estimate of the ramifications of the proposed 
change. Only in this way can the advanced logic of economic analysis 
be meaningful. 

If the effectiveness of the Navy's controlled maintenance program 
cannot be rationally measured, then the program will be that much more 
difficult to defend if criticized. 

The problem. The Bureau of Yards and Docks may be expected to be 
under increasing pressure to increase the effectiveness of the Navy's faci- 
lities maintenance program. In the immediate future, a means must be 
found for controlling and standardizing facility maintenance operations 
throughout the Navy under the concepts of the Dillon Report. Proper 
"feedback," measuring the effectiveness of the maintenance program at 
individual activities, and on a regional and Navy=-wide basis, must be 
developed. 

In looking to the future, it may well be that the implementation of 
recommendation 76 is only one step on the way to consideration of eventual 
consolidation of all Department of Defense facilities maintenance respon= 
sibilities into one office. In any "cost effectiveness” analysis of the 
wisdom of such a future change, the Bureau of Yards and Docks must be 
able to support in logical fashion with rational numerical data the advan- 


tages of the position in this matter that it desires to support. 





Even if these changes were not to develop, the Bureau of Yards 
and Docks, as the technical director of the facilities maintenance program, 
must continuously evaluate and improve the effectiveness of the Navy's 
maintenance program, striving to extract the maximum maintenance value 
from each maintenance doliar expended. Project THRIFT is just one 
example of the frequent pressures placed upon the Navy (and all services) 
to Squeeze more and more military value out of each taxpayer's input 
cies 

One major key to solving a problem is being able to measure its 
status. By being able to measure temperature, one can easily detect 
when the room or office temperature is either too high or too low in 
relation to a desired standard. Action may then be taken to change the 
existing temperature so as to conform to the standard. One of the basic 
needs of the facilities maintenance program is to have a "thermometer" 
which will measure the effectiveness of the program. The problem, then 
which this study is concerned with is the following: 

WHAT IS MEANT BY MAINTENANCE EFFECTIVENESS , 
AND HOW MAY IT BE MEASURED? 

The successful solution of this problem would be of great benefit 
in improving the maintenance program. Activities with poor effectiveness 
could be determined and appropriate funding or consultive assistance 
given. It would be easier to identify those activities which have 

 cineiintendent: U.S. Naval Postgraduate School memorandum 


of 15 January 1964, forwarding a reproduction of SECNAV message of 
11 January. 





inordinate maintenance resources available to them, and provide more 
rational criteria for diverting maintenance funds tc activities where the 
maintenance return per dollar would be greater. The maintenance level 
of similar types of installations within the Navy could be more stan- 
dardized, with a resultant maximization of maintenance benefit Navy-wide. 
The measurement device could provide important justification data in 
attempts to obtain additional maintenance funds and in competing with 
more glamorous programs for the funds which are now available. Pros 
and cons of future consolidations of maintenance effort between services 
or agencies within the Department of Defense could be approached 
from a more rational position than that of mere opinion based upon 
service loyalty considerations and intuitive arguments. 

Study. limitations. The problem of maintenance effectiveness 
measurement is studied in this paper from a conceptual point of view, 
and does not include complete details as to how any one system should 
be established. It is thought that the first need in analyzing the problem 
is to develop the basic concepts of why the problem exists, what is 
presently being done to solve it, what concepts might prove useful for 
further development and some specific recommendations as to future study 
to further the problem solution. This much has been attempted in this 
study. 

It is taken as a basic precept that the ultimate solution, or that 
which will be found to be the optimum solution, must be practical and 


workable at the local level, utilizing regular maintenance control 





personnel without advanced theoretical training. Theory may have to 

be modified for workability in some instances. In this event, the benefits 
of utilizing a "pure" theoretical system (even if one were to be found) 
wouid be secondary to developing a system which tells the manager in 


practical language the information he desires to know. 





CHAPTER ih 
UNDERSTANDING EFFECTIVENESS 

Fundamental to this study is a clear idea of what effectiveness 
is, and what concepts are necessary for its attainment. This involves, 
in turn, understanding of the concept of efficiency as distinguished from 
effectiveness, and the importance of the role of subjective, personal 
analysis, in determining effectiveness. A later portion of this chapter 
presents a conceptual outline of effectiveness requirements, indicating 
four basic steps to attain maximum maintenance effectiveness. 

1. EFFICIENCY - EFFPEGTIVENESe — oU SJrCluly ian 

Efficiency. Efficiency, as used in this study, is concerned with 
technological ability to achieve a certain stated output with a minimum 
of resource allocation. Once a decision has been made to repaint the 
exterior of a specific facility, it should be possible to develop a 
reasonably objective estimate of how long the job should take, how 
much material is required, and the cost of the project, given local wage 
rates and material costs. This estimate can be based upon the cbservable 
physical characteristics of the specific facility. 

Once the job has been completed, comparison of actual costs in 
terms of money, labor, and material usage may be used for efficiency 
indicators for the painting of the facility. Comparison may be made with 
similar data obtained from other activities to further indicate how effi- 


ciently the job has been done, and, broadening the scope, how efficiently 





the maintenance work done at one activity has been done in comparison 
with how well maintenance work has been done at another activity. 


The Engineered Performance Standards concept is a primary tool to 





control efficiency of maintenance tasks. Certainly the maximization of 
efficiency is one prerequisite to effective operation. Any procedure which 
contributes to maintenance efficiency, such as the use of Engineered 
Performance Standards, must be supported and utilized. 

The data used in measurement of efficiency is primarily of an 
objective nature. Tne job or work performed can be described in explicit 
detail, particularly once it has been completed. The costs involved are 
merely matters of historical fact. The method used may be described in 
a Similar objective, impersonal manner. Nevertheless, the problem of 
subjectivity still creeps into the efficiency problem. 

Standards for performance will vary from activity to activity. The 
problems of exterior painting may be relatively minor in Southern Cali- 
fornia, but extreme in Adak. Costs cannct be directly compared between 
activities having different wage rates or material procurement advantages 
or disadvantages. Who is to say, in a purely objective manner, what are 
proper standards for a particular location? At best these may only be 
determined historically based on what costs have been in the past to 


perform certain work. Standards are not necessary based on what it 


nll Cytmoediy | 
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should cost to perform the required work. 
Ever since the inception of the Navy's controlled maintenance 
program, attempts have been made to determine which data would be 


y 





most valuable in helping evaluate the performance of maintenance 
organizations. A summary of the work in this field by private industry, 
illustrated by Chapter III of this study, shows that industry is also 
concerned with the problem of evaluating maintenance performance. Of 
particular recent note, however, are experiments at the Navy's Civil 
Engineering Laboratory, using Operations Research methods, to gain 
further insight as to what data or characteristics are important in 
1 
measuring performance. Although the Navy has come a long way in 
improving methods of determining efficiency, the best answer probably 
has still eluded all efforts and the present concept of Research and 
Development work towards this end is to be encouraged and fostered. 
Effectiveness and military worth. Effectiveness, as used in this 
study, deals with the amount of total value or military worth gained by 
the Navy as a result of utilization of resources for maintenance of Naval 
shore facilities. The measurement of effectiveness would, in some 
manner, indicate how much military worth was gained, perhaps in 
] . — 
A partial list of studies undertaken through the Naval Civil 
Engineering Laboratory is as follows: 
a. Factors in Analysis of Maintenance Costs (Task Y-5015- 
15-06-504) 
b. Study to determine methods of distributing maintenance 
resources (Task Y~F015-15-06-506) 
c. Effect of EPS and MME Rating on Backlog of Essential 
Maintenance (Fask Y-F015-06-508) 
d. Factors affecting Utilities Consumption and Costs 
(Task Y-F015-15~06-509) 


e. Manpower Input Standards for Transportation Equipment 
(Task Y-F015-15-06-~510) 
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comparison with how much might have been gained by allocation of the 
same resources to other projects. Military worth is simply the ability 
of a particular project or alternative to provide tangible or intangible 
benefit to a required military capability. Any project having potential 
military benefit, whether to help morale, restore power service, or to 
unplug a drain, would have some degree of military worth. The concept 
of military worth is directly analagous to the economic concept of con- 
sumer utility. The keystone to measurement of effectiveness will be 
shown to be the problem of the measurement of expected military worth 
to be generated by a particular project. 

The first requirement for effectiveness is that the maintenance 
effort be efficient. Each project which is undertaken should be accom- 
plished with the minimum of resources necessary for the job's satisfactory 
completion. 

In addition, however, it must be determined that each job under-= 
taken was the right job to do at the right time. This requires the main- 
tenance program to be aligned primarily with tne operational needs of the 
Navy and secondarily with the needs of the maintenance program, where 
the two happen to be in conflict. This may mean that less than com= 
pletely efficient maintenance of facilities may be most effective in 
station support. Diversion of maintenance resources to optimum align- 
ment with station mission needs may result in wot being done which 
fosters the mission of the activity, but which may detract from actual 


maintenance of station facilities : 
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The need for the maintenance effort to be aligned with Naval 
Mission requirements brings to light the primary measurement problem. 

It will be seen from a review of Chapter III of this study that commercial 
enterprises can obtain a fairly direct evaluation of their maintenance 
effectiveness by noting the effect on overall production costs of increased 
(or decreased) maintenance effort. In this case the goals of the company 
and the resources of the maintenance program are expressed in the same 
measurable units - money. In the Navy, the maintenance costs are 
eaSily measured in dollar terms, but the goals of the organization are 
difficult to define in any specific, measurable terms, much less a dollar 
figure. Who can define a unit of "readiness," or "capability?" How much 
money is one unit of "readiness" worth? How many units of "readiness" 
are produced by any one particular maintenance project? What is the 
standard of military worth which should be met by any valid maintenance 
project? 

Subjectivity. Here is a true dilemma. The need for objective 
measurement of maintenance effectiveness is fairly obvious, but this 
needed measurement is dependent upon subjective criteria. The very 
definition of subjectivity would seem to make objective measurement of 
subjective qualities impossible, at least in the scientific sense. 

If one were to assume, at least for the moment, that objective 
measurement devices are incapable of measuring the subjective valua- 
tions of military worth necessary to know the effectiveness of the 


-~ 


facilities maintenance program, then the next logical step would be to 


ne 





determine whether measurement approximations might be utilized which 
would approximate measurement of military worth well enough so that 
more rational decisions may be made regarding the problem than would 
otherwise be the case. There are a host of examples where this is done. 
School examinations, fitness reports, public opinion polls, intelligence 
tests, and thousands of other measurement devices are used every day 
to obtain some valuation of subjective traits. These valuations may not 
be perfect, but they do at least provide a workable substitute in many 
cases. Cay some errors are made. Students can always recall 
instances where they feel they should have gotten higher (or lower) 
grades. Some Naval officers are unfairly passed over (others unfairly 
selected) each year. The system, however, operates with more validity 
than would be the case without these devices. 

Perhaps, if military worth is not susceptible of direct meaSurement, 
the various subjective components of military worth such as mission 
effect, morale effect, appearance effect, readiness effect, etc. could 
be described in familiar terms so that some numerical value could be 
attached to these individual segments. The segments could be combined 
in some fashion and some sort of composite numerical valuation of mili- 
tary worth approximated. If nothing else, an attempt to make this type 
of analysis might lead to a better understanding of just what the factors 
are which are important in evaluating military worth. 

Whether or not any sort of objective system is developed to measure 


subjective criteria, subjective ranking of programs and projects in some 





preference order must continue to be made in the normal course of 
business. These decisions are being made now and have always been 
made. Recommendation 77 of the Dillon Report is one attempt to improve 
the decision process for facilities management. 

Before completely ignoring the possibility of measuring military 
worth by objective theoretical means, some comments of von Neumann 
and Morgenstern concerning the ability of evaluating economic “utility” 
should be mentioned. They note that the problems of evaluating "utility" 
are reminiscent of the problems with first attempts to measure heat, based 
on the intuitive concept of one object feeling warmer than another. The 
reader is cautioned against making any final claims with regard to the 
impossibility of developing a more precise utility theory. The theory of 
heat may repeat itself, and nobody can foretell with what ramifications 
and ie Certainly it would be wise to stay abreast of develop- 
ments in current economic theory in order to improve solution of subjec- 
tive measurement problems. Just as it is impossible to presently prove 
that utility or military worth can be objectively measured, so it is impos- 
sible to prove that at no time in the future will such a development be 
possible. The future must speak for itself. 

Il. FOUR STEPS \ TO EFPECIIVEMEeSs 

Having described the basic problems in measurement of effective- 
ness and explaining the basic terms involved and their nature, itis 

2Tohn von Neumann and Oskar Morgenstern, Theory of Games and 
Economic Behavior (second edition: Princeton: Princeton University Press, 
1947), pp. 16-17. 
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possible to outline four basic steps which must be taken in order for 
the Navy's facilities maintenance program to be considered effective. 
These four.steps are the following: (1) Determination of deficiencies, 
(2) Determination of the relative priority of deficiencies, (3) Performing 
projects efficiently, and (4) Proper resource allocation between activities. 
Determination of deficiencies. The starting point for any mainte- 
nance project is the determination that a particular situation is deficient 
in some respect. Mui deficiencies in the Navy are determined by 
complaints of personnel occupying the spaces involved or by routine 
inspection by Public Works personnel. NAVDOCKS P=-322 outlines the 
Department of Defense criteria for judging the desired level of maintenance 
for the various types of structures owned by the Navy. This guidance is 
used in evaluation of reported deficiencies in order to determine whether 
the deficiency is, indeed, a valid one which requires correction. 
Subjective considerations are very much a part of the determination 
of deficiencies. There is no single point in time at which an alarm bell 
can be set to ring "Now is the time to paint building 5-105" with any 
rational meaning. One determines whether a building requires a coat of 
paint or not by looking at it and inspecting the condition of the structure. 
A personal interpretation is required before the decision can be made as 
to whether the building requires the paint or not. The NAVDOCKS P-322 
guidelines are purposely general in nature, in order to allow flexibility 
at the field level. This is as it should be. The point to be made here 


is that any system of measuring maintenance effectiveness must also 
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include some evaluation of the effectiveness of the manner in which 
deficiencies are determined, and the relative validity of these defi- 
ciencies. This is one of the subjective areas which must be considered 
for a complete analysis. 

Determination of the relative priority of deficiencies. This really 
is the problem referred to earlier of making sure that the projects under- 
taken are the ones which should be done. This is always a large problem 
for the Public Works Officer, due to continual pressures from all sides to 
divert maintenance resources to where they may benefit a particular office 
or department to the maximum, but not necessarily where these resources 
may best serve the general interests of the Navy. So much effort may be 
expended in caring for fairways and greens on the golf course, that it is 
difficult to keep the grass around the runways under control. Who is to 
say just is the dividing line is between how much attention the adminis- 
tration building should get versus the supply building, or between the golf 
course, the Commanding Officer's yard, or the runway areas. The problem 
here is deciding which jobs will return the most military worth per dollar 
spent so that those projects may be accomplished first. Chapter IV of 
this study proposes a concept for solution of this problem. 

Performing projects efficiently. This problem is the one most 
familiar to the controlled maintenance program. Its solution involves 
the proper engineered solution to maintenance problems; proper allocation 
of men, materials and funds to the projects which have been assigned to 


the shops; and control over sick leave, production losses from delays in 
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transportation, and similar problems. This is the characteristic of 
effectiveness which can be most easily measured, through comparison 
of objective cost accounting data against pre-determined standards. 
The establishment of the standards, however, has some subjective 
considerations involved, as has been previously noted. 

It should be remembered that an activity may perform very effi- 
ciently without being highly effective. By doing only projects of relative 
Simplicity, or where there are a minimum of cross-scheduling problems 
between work centers, projects may be done very efficiently. If these 
projects being accomplished are not the projects which should be accom- 
plished to meet the mission of the activity and the Department of Defense 
maintenance criteria, then the operation will not be effective. 

Proper resource allocation between activities. Even though each 
Naval activity is efficient in its maintenance program, and performs those 
projects which return the most military worth per dollar cost, the overall 
Naval maintenance effectiveness will not be maximized unless resources 
are allocated between activities in a balanced manner. An activity with 
only meager maintenance funds would be able to apply these funds only 
to those projects with the highest military worth return. An activity 
with inordinate resources would be able to accomplish projects of 
lesser military worth. The overall Naval maintenance effectiveness 
would be improved if some of the funds from the second activity were 
transferred to the first activity where the military worth per dollar 


expended would be higher. This action would increase the total sum of 
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military worth gained by the Navy's maintenance budget. An optimum 
system for measurement of maintenance effectiveness would contain 
some device to evaluate the level of maintenance being attained at each 
activity in order to spot where redistribution of resources should be made 
in order to maximize benefit to the Navy. Such an indicator must be 
capable of withstanding violent attack by representatives of the manage- 
ment bureaus and commands adversely affected by its use. 
SUMMARY 

Differences in concepts of efficiency and effectiveness have been 
discussed in this chapter as well as the major role played by subjective 
analysis in measuring the variables involved in effectiveness measure- 
ment. Four basic steps to be taken in obtaining maintenance effective- 


ness within the Navy have been outlined. 
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CHAPTER III 
MAINTENANCE OUTSIDE THE MILITARY 

This chapter discusses the status and concepts of controlled 
maintenance programs outside the realm of the military, and illustrates 
program concepts which could assist in increasing the effectiveness of 
Naval maintenance. Current developments in private industry have been 
summarized first, followed by a summary of concepts and programs 
developed by public agencies with the state of California. 

I, BASIC INDUSTRIAL CONCEPT 

The key to the industrial concept of plant maintenance is the 
shifting of maintenance costs from their traditional niche as general 
overhead costs to direct product costs. The goal no longer is lowest= 
cost maintenance, but lowest-cost product = The measure of effective- 
ness of the maintenance program is in the same units as the cost of the 
program ~ dollars. The problem of evaluating the effectiveness of the 
maintenance program can then be reasonably well resolved by data of an 
objective nature gathered through the company’s cost control system. 
As may be seen by a review of the literature summarized in later sections 
of this chapter, the methods of utilizing this data are varied and numerous. 

Il. USE OF CONTROLLED MAINTENANCE PROGRAMS BY INDUSTRY 

There is still less than complete use of controlled maintenance 

systems within private industry, despite the time-proven advantages 


le . G. Wyder, “Where are we Headed in Maintenance?", in 
Factory CXXII January, 1964), 68-71. 





of such systems. In arecent review, it was shown that many mainte- 
nance managers are not using proven cost-cutting methods and some 
had not even adopted basic controls i In an editorial in Factory magazine, 
it was pointed out that many companies do not pursue their maintenance 
control potential to the fullest. A good deal of attention has been paid 
to review of the larger jobs with little analysis of the small jobs. In 
one plant a team of two engineers was reported to have accumulated 
savings of $400,000 over a period of eighteen months, out of an annual 
budget of $2.5 million, by review and analysis of small maintenance jobs 
costing less than $100.° 

Despite these occasional editorial reproofs, a reasonable review 
of the available literature on industrial controlled maintenance systems 
will show that there are a great many new concepts and procedures which 
are of the highest sophistication and represent a great deal of high-level 
analysis and development. A short review of several of the more interest- 
ing ideas noted during the literature review constitutes a later portion of 
this chapter. Of particular note is the increased importance of the 
maintenance function when an industry has shifted to more automated 
production techniques. Atypical article notes that as machines handle 
more complicated tasks, fewer human hands are needed to set them in 

2"Get Maintenance on the Move,” in Factory CXXI(September, 1963), 
pp. 96-7. 

- R. Bittel, “Molehill Mountains in Maintenance," in Factory 


CXXI (January 1963), py. 41. 
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motion, and required human production skills decrease. At the same 
time, however, the number of hands necessary to keep these machines 
in Operation will increase, the maintenance skills must increase to keep 
pace with the advanced machine technology, and the pressures for im-=- 
mediate service and repair are extreme due to the high cost and key 
importance of the automated equipment. 

A reasonable conclusion would be that the maintenance effort will 
become more and more critical to profitable operations, and that a great 
deal of advance in maintenance concepts and technology will be developed 
in the future by private industry. The next portion of this chapter contains 
a review oi the more interesting concepts noted during the review of the 
literature for this study. 

HI, RECENT INDUSTRIAL MAINTENANCE INNOVATIONS 

ocheduling by Mathematical Programming. An article by Wagner, 
Giglio and Glaser has reported the results of three experiments which 
explored methods for scheduling preventive maintenance by means of 
mathematical programming. The report states that the task of scheduling 
preventive {routine or planned) maintenance involves specifying dates at 
which manpower is to be allocated to an overhaul! of maior functional 
elements. A typical objective is to accomplish required maintenance on 
a timely basis, keeping to a minimum fluctuations in the aggregate amount 
of manpower needed. 

Three experiments were run, the first two being small scale 


laboratory experiments. The first experiment assumed 8 fictional 
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projects to be scheduled over a 10-week period. A table of manpower 
requirements and sets of possible starting dates were postulated which 
could lead to 5,184 possible schedules. The second experiment was 
Similar to the first, but postulated personnel requirements and possible 
starting dates such that 150,000 possible schedules could have been 
developed. Computerized schedules were developed for both of these 
problems. The third test involved a full-scale test at an industrial 
plant where 890 different projects extending over 13 months were to be 
scheduled. The total number of possible schedules exceeded (42) 1068, 
A schedule had been manually prepared by the plant maintenance planning 
staff for comparison purposes. The computer schedule was developed 
in 30 minutes on an IBM 7030 computer and required only 10 minor manual 
shiftings in projects. Seventy-eight percent of the projects were scheduled 
within one month of target date by the manual schedule Ninety-seven per- 
cent of the projects were scheduled within one month of target date by 
the computer. Manpower leveling was also improved by the computer 
Eenedtion 

Ratio Analysis. Most maintenance programs are concerned with ratios 
of various types. The most comprehensive list of these various effective- 
ness" ratios found was that contained in an article by Victor Z. Priel. 

OH, M. Wagner, Richard J. Giglio and R. G..Glaser, "Preventive 


Maintenance Scheduling by Mathematical Programming," in Management 
Science X (January, 1964) 316-334. 
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The ratios will not be listed in this study for lack of space, but are 
divided into three major areas as follows: 

a. Maintenance Effectiveness: Is maintenance manpower being 
fully utilized, and are its efforts effective? 

b. Amount of Maintenance: Is the service it's providing adequate 
for the plant? is 

c. Cost of Maintenance: Are the costs of maintenance commen- 
Surate with the results Sououe 

Critical Path Network Techniques. The increasing use of network 
techniques is suggested by another factory article. The article contains 
a good basic summary of "Network Management Techniques,” their advan- 
tages, problems and references to other articles concerning the technique. 
Of particular interest were six network examples attached as a fold-out 
to the basic article illustrating basic uses of these techniques a The 
advantages of the critical path techniques are likely to become more and 
more well~known to industry. 

Maintenance Gaming. A maintenance business game has been 
developed to provide insight for participants into some management 
problems of an organization which has the responsibility for servicing 
production equipment. The game was developed so that computations 

Victor Z. Priel, "20 Ways to Track Maintenance Performance," 
in Factory CXX, (March, 1962), 88-91. 

7Marvin Flaks, et. al., “Network Management Techniques" in 


Factory CXXII, (March, 1964), 86-94. 
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could be performed by hand or with the aid of a desk calculator in order 
that it could be played without the aid of a high speed computer. The 
game can be introduced in a college level course or management develop- 
ment program and completed in two hours. The game provides examples 
of problems in economic "trade-offs", iterative or trial-and-error problem 
solutions, waiting line phenomena, and insight into the techniques of 
business oe While apparently aimed toward the development 

of general business managers the game should have illustrative value 

to maintenance managers. 

Maintenance Prevention (MP). Maintenance Prevention (MP) is a 
phrase Factory magazine has coined to represent what is really a mainte- 
nance-oriented design concept. This concept seeks to minimize, through 
design considerations, the amount of maintenance required by a machine 
component, the whole machine, a process, a utility service, or a building. 
Possibly an entire unit may be eliminated through some design change. 
The elimination of a unit eliminates the maintenance of that unit. In 
short, the concept willingly accepts higher design or initial equipment 
cost where the maintenance savings from the use of the higher quality 
equipment will repay the larger investment in future lower operating 


9,10 
and maintenance costs. ’ 


Ep omeet Campbell, Donald Pierce, and Paul E. Torgersen, "The 


Maintenance Game," in Journal of Industrial Engineering AV, (January- 
February, 1964), 30-6. 


earl G. Wyder, “The Expanding Role of Maintenance,” in Factory 
CXX (April, 1962), 105. 


aU etnrenenes Prevention is Growing Up, 
(January, 1963), 68-71. 24 
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Predictive Maintenance. Predictive Maintenance involves the use 
of sensing, measuring, or control devices to determine whether there have 
been significant changes in the physical condition of equipment. Various 
visual, audio, electronic, pressure, thermal, etc. devices may be used 
for periodic inspection of equipment in order to determine major change 
in condition. The idea is not to dismantle equipment for overhaul unless 
there is some indication that the overhaul is actually eee a 

Another example of this technique can be seen in the “Selective 
Maintenance” program of General Electric. The prosram results in a 
“Reliability Index" being developed for the types of equipment under con- 
sideration. The reliability index for motors and generators is based upon 
an analysis of factors such as results of electrical tests, visual inspection, 
age, motor environment, and types of duty. Each of the factors is assigned 
a numerical value in accordance with a weighting system established for 
the motor type. The composite of these weighted evaluated factors is the 
reliability index for the particular motor under analysis. The motors which 


have been assigned the least desirable reliability index are then the ones 


which are overhauled first. Ae 


lliames D. Quinn, "The Real Goal of Maintenance Engineering, " 
in Factory, CXXI (June, 1963), 90-3. 


I2p . L. Hatschek, "Set Maintenance Priority with a Scorecard, " 
in Mill and Factory LXXII (April, 1963), 62-5. 
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Effectiveness Factors. One use of weighted factors for an over-all 
maintenance valuation of an electric motor has been mentioned in des- 
cribing the Predictive Maintenance concept. Another use of weighted 
factors to measure maintenance effectiveness is described in an article 
in Mill and Factory. In this General Electric procedure eight different 
maintenance indicators were assigned weight values with maximums 
from 5 to 35, the standard total being 100. Where standards are exceeded 
the overall evaluation may exceed 100. The factors and their respective 
weights are as follows: 

a. Cost of direct labor actually spent vs. budgeted - 35 

b. Other controllable expenses vs. budgeted - 25 

c. Schedule Effectiveness: number of hours actually 

spent on construction and repair work orders 

against hours scheduled for the jobs = 15 
d. Cost Reduction: monthly savings against the target - 5 
e. Downtime Attributable to Maintenance: actual 

machine downtime vs. a standard (for mechanical 


trouble) - 9 


Ot 


f. Call-in Hours Paid: compared to a standard - 

g. Manufacturing losses attributable to maintenance: 
dollar value of scrap caused by maintenance failure 
against a standard - > 

h. Backlog trend: number of jobs received against 
those finished = | 5 


Total 100 





The numerical valuation for maintenance effectiveness determined 
in this manner was used as an index and trend of maintenance opera- 
tions. ae 

Plant Lighting. One approach to the problem of replacement of 
burned-out lamps is summarized in an article in Supervisory Management. 
At one plant a regular replacement oi all lamps in an area is carried out 
when the burn-outs, including those which have been replaced since 
the last general replacement, reach 20% of the total. At this time all 
fixtures are also cleaned and serviced. Between general replacements 
burned out lamps are not replaced except in areas where necessary to 
maintain certain required lighting intensities. A certain number of the 
replaced lamps which are still functioning are retained as replacements 
for areas which require some replacements but which are due for general 
replacement in the near future. The system avoids costly trips and setup 
time to change lamps individually. ue 

Communications. At one large, multi-storied plant, additional 
responsiveness and maintenance savings have been accomplished through 
the use of radio-controlled maintenance organization. The plant has ap- 
proximately 40 acres of floor space containing many different production 


divisions and diversified product line. Little of the equipment is new 


Pored Metaxas, “Measuring overall maintenance Efficiency,” 


in Mill and Factory LXIX, (December, 1961), 84-7. 


14"pjant Lighting: The Advantages of Planned Maintenance," in 
Supervisory Management IX (April, 1964), 54-6. 
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and there is much emergency maintenance required. To alleviate this 
problem, four field foremen, each generally responsible for one plant 
floor, inspect and estimate each job before work begins and stay with 
the job whenever their help is needed. All paperwork is done at a central 
headquarters, which controls the operation through 2-way personnel- 


carried radios. 15 


Summary of industrial innovations. The cited examples illustrate 
the level of sophistication and development of maintenance concepts in 
private industry. Although many concerns do not make adequate use of 
advanced controlled maintenance techniques, it is equally obvious that 
many other concerns have advanced to a very high conceptual state. 
Future developments in the industrial family should be continually 
monitored for possible application to Naval needs. 

Most of these developments deal with improving the efficiency 
of maintenance operations. Very few deal with the problem of determining 
whether the work being done is actually the work which should be done 
considering all of the circumstances. Mr. Quinn's article notes that 
doing the maintenance job more efficiently seems to be the principal 
objective of many maintenance engineers; but worthy as this objective 
is, a basic question is being overlooked: "Is the work being done 


really necessary?" Quinn notes that even with an operational controlled 


1S uRadio Expedites Central Control of Maintenance, 
CXXII (January, 1964), 72-5. 


in Factory 
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maintenance system, there are frequent cases where inadequate 
analysis of the need for all work being performed is carried out. 16 
Despite the advancing eteloar, and improvements in project efficiency, 
the literature of private industry offers no magic — for the problem of 
determining which projects are the right ones to be done. The work may 
be efficient, but not necessarily as effective in contributing to profit- 
ability as it should be. The Naval maintenance program has the same 
effectiveness problem, but with the added handicap of the lack of an 
overall "profitability" indicator. 
IV. MAINTENANCE WITHIN STATE AND LOCAL AGENCIES 

Decentralization. From correspondence received it is evident that 
decentralization is a predominant feature of the maintenance efforts of 
public agencies. An initial letter was addressed to the State Auditor 
General asking for information on criteria used by the State of California 
in evaluating the State's maintenance program. The reply stated that the 
Department of General Services was the agency having the principal 
responsibility for State buildings and grounds, but that this responsibility 
was decentralized to a high degree. It was suggested that several other 
specific agencies be contacted separately to obtain information on main- 


tenance within their areas. 17 


a olan > logmaaeiie 


17State of California, Office of the Auditor General, letter of 
February 3, 1964, signed by William H. Merrifield, Auditor General. 
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The Department of General Services stated that the State of 
California had no prepared summary of the State's facilities maintenance 
policies or specific criteria for budgeting or evaluating the maintenance 
"product." The opinion was expressed that expenditures for maintenance 
of physical facilities can only be standardized when dealing with two or 
more installations which are practically identical. It was thought nece- 
ssary to review each individual installation to provide the proper resources 
for adequate maintenance The proper level of maintenance was con- 
sidered to be a matter of judgment which could not be resolved soon 
enough by any formula to convince budget analysts of the necessity for 
additional maintenance funds. It was further stated that some time ago, 
the State Division of Architecture had a special crew which visited all 
State institutions and recommended maintenance projects and programs. 
However, it had been determined that the expenses of such a staff were 
not warranted for the value obtained and that it would be better for the 
individual institutions and their headquarters offices to establish their 
own maintenance programs. 18 Replies from the Department of Finance, 
the Joint Legislative Committee, California Legislature and the Division 
of Highways further indicate decentralized operation and establishment 


of localized criteria for effective maintenance effort. ae 


18state of California, Department of General Services, letter of 
March 30, 1964, signed by A. W. Collins, Assistant Director. 


lSstate of California, Department of Finance, letter of March 193, 
1964, signed by F. J. Murphy, Senior Budget Analyst. 


Z0state of California Joint Legislative Budget Committee, California 
Legislature, letter of March 13, 1964 signed by A. Alan Post, Legislative 


Analyst. 
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The Division of Highways does review centrally the results of each 
district with the objective of maintaining a uniform standard through- 
out the state for similar road and traffic conditions. Close control of 
highway maintenance funds is maintained by the Maintenance Engineer 
in Headquarters. Any work other than routine maintenance must be 
specifically authorized and a work order allotment issued prior to the 
work being undertaken in the field. In this category are items of major 
storm damage repair, preventive maintenance measures, bridge repairs 
exceeding $1000 per bridge, and similar activities et The operation 
of the Division of Highways seems to more closely approximate that of 
the Bureau of Yards and Docks than any of the other State agencies 
from which replies were received. 

Ratio Analysis. Those few agencies which replied most specifically 
to the question of how they measured their maintenance effectiveness 
placed most emphasis on ratio analysis. The Division of Highways plans 
for numbers of maintenance personnel on a historical basis. The budget 
is then based on the personnel level, applying a factor for the cost of 
the maintendnce operation per man, including necessary equipment and 
materials to perform all normal maintenance. To equalize the work load, 
the local organizations are compared on the basis of the number of lane 
miles maintained per man. In addition, the number of acres of land- 
scaping maintained per man is also considered. Some other features 


eerste of Califomia, Division of Highways, letter of March 3, 
1964, signed by C. J. Brown, Asst. Maintenance Engineer. 
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are considered, however, such as the number of electrical safety 
devices or steel structures which may require special maintenance 
operations. F 

The University of California compares maintenance costs in 
relation to outside gross square footage of building area between 
various campuses of the University, making an analysis to evaluate 
deviations due to local conditions. As a point of interest, reference 
was made in their letter to BUDOCKS publications for obtaining further 
information on yardsticks and standards 28 

The County of Los Angeles stated that it had no schedule of 
maintenance standards to provide, but that for the past several years 
maintenance of new space had been budgeted at an average of 20¢ per 
square foot. It was further stated that one custodian normally services 
about 8000 square feet, although cther formulas for staffing of custodial 
services had been worked out and could be made available upon requectaae 

Although the letter requesting information from these agencies was 
not phrased in such a way as to obtain detailed data on the current 
status of implementation of a controlled maintenance program at each of 


the agencies, the general impression from review of the correspondence 


received was that the decree of control over the public agencies 


22thid, 


23yniversity of California letter of March 25, 1964, signed by 
S. A. Musser, University Engineer - Maintenance and Operations. 


Z4 county of Los Angeles, Mechanical Department, letter of 
March 4, 1964 signed by A. L. Journey, Assistant Chief Deputy. 
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maintenance effort was not equal to that of the Naval program, nor of 
the level of sophistication which has been developed in portions of 
private industry. Perhaps this has been due to the lack of centralized 
review of maintenance costs and effectiveness, and to the ever-present 
public service problem of the lack of a "profit” criterion by which the 
effectiveness of the organization may be judged. It was refreshing to 
note, however, that the City of Los Angeles outlined the development 
of a program to determine quantitative evaluation methods for building 
maintenance and operation. Comparative effort values for similar work 
are soon to be a part of their cost and management reports. Recently, 
the introduction of task-area units into the cost-keeping system produced 
their first task-unit cost report. The annual cost per square foot for 
plumbing systems maintenance for a given building is now available, 
as are similar unit costs for other maintenance and operations tasks. 
The planned periodic inspection program has not, however, yet been 
implemented. = The obvious intent of the new program will be to develop 
a well-controlled maintenance system. 
V. SUMMARY 

Private industry, and to a limited extent, some public agencies, 
have developed many sophisticated innovations in improving the effi- 
ciency of their maintenance efforts. These developments should be 
monitored continually by the Bureau of Yards and Docks. Very little 

25City of Los Angeles, Department of Public Works, File Number 
550.0-Collins letter of February 26, 1964 signed by W. R. Blakely, 
Director, Bureau of Public Buildings. 
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information, however, was found which bears upon the problem of 
measurement of effectiveness of the maintenance effort where the 
profit motive is lacking. The concepts of Predictive Maintenance 
and Maintenance Prevention do bear directly upon the Navy's etfective- 
ness problem. At least one example of “measuring” effectiveness with 


the aid of a weighted evaluation system was noted. 





CHAPTER IV 
A PROPOSED USE OF MILITARY WORTH CONCEPTS 

The problems of measurement of military worth have been dis-= 
cussed in Chapter Il. In addition, some comments of Oskar Morgenstern 
are appropriate here. He has noted the need for more research by the 
Department of Defense in the area of military worth determination. The 
development of a more workable method of determining military worth of 
various alternatives would be more important than to introduce some 
further, purely technological advance in some weapons system. It 
would improve the use of all of them. : 

I. PHE CONCEPT 

It may be recalied that two of the four steps necessary for effec- 
tive maintenance operation involved allocation of resources. One decision 
was at the activity level in determining which projects to perform; the 
other was at the Bureau or Field Engineering Office level in deciding 
where best to allocate resources between activities. The position taken 
in this study has been that the allocation decision should be based on 
the determination of which alternative provides the greatest return in 
military worth per dollar spent. 

If the military worth, or a suitable approximation thereof, for 
each alternative could be determined on @ numerical scale, the allocation 
problem would be basically solved. One would calculate the military 


] a — . 
Oskar Morgenstern, The Question of National Defense, 
(New York: Random House, 1959), pp. 203-5. 
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worth of each project, divide each military worth figure by the respective 
estimated total cost of the project, and do those projects first which 
provide the highest military worth per dollar expended. 

One immediate objection to this theoretical concept comes from 
the association of the concept of military worth with the economic 
concept of consumer utility. In order for the military worth valuations 
to be meaningful for preferential ordering when divided by the project 
cost, the valuation of military worth must possess cardinal ordering 
properties. In order for a $1000 project to return the same military worth 
per dollar expended as a $500 dollar project, the valuation of military 
worth for the $1000 project must be twice that of the military worth 
valuation of the $500 project. The most accepted economic utility 
theory holds that consumer utility preferences cannot be expressed by 
cardinal scales, but only by ordinal preference. One can Say that a 
certain project is "better" than another one but it is not rational to say 


that one project is twice as "good" as another one, since "utility" is 





an abstract term with no measurable properties. Utility is really a 
name given to a subjective concept to explain observable behavior of 
individuals and their market choices < 

Another problem might develop if attempts to use such a system 
are broadened to use by more than one individual. This might be the 


2Por a more complete review of contemporary utility theory and an 
excellent bibliography for further study see: Ernest W. Adams, “Survey 


of Bernoullian Utility Theory," Mathematical Thinking in the Measurement 
of Behavior, Herbert Solomon, editor, (Glencoe, Illinois: The Free Press, 


1960), pp. 151-268. 
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case if a system developed for use at one activity were to be adopted 
by all activities within the region of a Field Engineering Office, the 
system then to be used in evaluating needs of one activity versus 
another in terms of the expected military worth for available resources. 
This problem stems from the inaoility to transfer meaning of one person's 
“worth” or “utility” to another person. Individuals have different criteria 
of value, and comparisons of different utility valuations by different 
people are illogical. Criteria for one activity commander will appear 
different from that of another activity commander, and valuations of 
military worth may have been based on different command policies. 
Individual criteria which may work well at a particular activity may not 
be appropriate for inter-activity comparisons. 

There is, however, far from universal agreement that “utility” 
is solely an ordinal concept. The classical contemporary discussion 
on this point is contained in a text by Von Neumann and Morgenstern. 
If an individual prefers alternative C to A and prefers aieecenye Ato B, 
but prefers A to an alternative situation where there is a 50% probability 
of getting B and a 50% probability of getting C, a plausible argument 
may be made that his preference of A over B is in excess of his preference 
of C over A. Determination of the relative combination of probabilities 
of getting B or C for which the individual] claims to be indifferent to an 
alternative choice of A may provide some criterion for measuring the 


relative preference of the individual for B and C in comparison with A. 


Sohn von Neumann and Oskar Morgenstern, Theory of Games and 
Economic Behavior, (second edition; Princeton: Princeton University 
Press, 1947) pp. 8-20. 
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From another point of view, it is felt that if an individual states he is 
undecided whether to buy two items of article D or one item of article 
E with the same resources, then one could do somewhat better than to 
say that the individual merely places his preference for one article of 
E above that of one article of D. 

In answering the problem of inter=transference of “utility” or 
“worth” values, it should be noted that the concept of military worth 
has one significant difference from that of consumer utility. Military 
worth evaluations should not be personal in nature. They should be 
based on high level policy guidance, common to most, if not all, acti- 
vities. There are still problems in using a common military worth system 
for allocation of resources between activities, but these are more mecha- 
nical than theoretical. Basic decisions at all activities should be in 
the best interests of the Navy. Additional comments on this subject 
are placed near the end of this chapter. 

From a realistic viewpoint, any sort of solution to the problem of 
measurement of utility or military worth in the near future will have to 
be satisfied with an approximation to a final mathematical, objective 
solution. The theory just has not advanced to the state where one 
person's utility preferences can be precisely measured in numerical 
terms for any predictive purposes. This study takes the position, 
however, that a decision model could be developed in the interim, 
utilizing cardinal properties of a military worth scale, which could 


assist the decision-maker, on at least the local level, in his decisions 
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as to where best to allocate his maintenance resources. While not at 
all theoretically or mathematically pure, this system would provide 
numerical scales for certain factors important to the decision. Values 
for these factors could be combined in some designated manner so as 
to reach a derived numerical approximation for the military worth to be 
expected from the alternative under consideration. The approximation 
of the military worth of the project could then be divided by the total 
cost of the project to determine the military worth of the project per 
dollar expended. Projects contributing the most per dollar to the military 
worth of the Navy would be done first, and those projects which have 
the least return would be done last, or not st all. 

Although this system cannot be completely supported in an 
objective theoretical way, it at least is an attempt to develop a system 
which would work in the manner in which the decision-maker would want 
such a system to work, and perhaps in the way in which the decision= 
maker should think pending the development of a completely pure 
theoretical model. 

Il, THE SYSTEM 

Such a system has been developed by this study, using rather 
arbitrary factors. Two small introductory experiments were conducted 
in an attempt to see if such a system, developed through more detailed 
analysis, might be of service. The complete description of factors, 
their weights, computational steps, and other procedures of these 


experiments are contained in Appendix 1. The basic concept and 
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general outline of procedure is covered here. 

Two sets of factors were used. The first set consisted of two 
so-called "Qualitative Factors," which were graded without consideration 
of the scope or cost of the projects. As actually used, these factors were 
intended to help answer the questions: (1) "To what extent is the proposed 
work aligned with the mission needs of the activity?", and (2) "To what 
extent will the cost of the project be returned during operation of the 
facility involved?" 

The second set of factors consisted of three "Quantitative Factors," 
which were intended to provide a means to adjust the military worth score 
for the magnitude of the benefits to be realized for each alternative project. 
These factors were intended to help answer the questions: (1) "How much 
of the mission of the activity is affected by this project?", (2) "How 
great are the expected opportunity cost savings in absolute magnitude?", 
and, (3) "To what extent is the project postponable without loss of major 
benefits ? " 

Specific guidance for grading each factor is contained in Tables I 
and II, Appendix I. 

Emphasis was placed on keeping the evaluaticn effort simple 
and concise, although some pure theory may have been sacrificed in 
so doing. For example, no attempt was made to determine costs beyond 
that required for the initial work. However, the personnel acting as 


evaluators during the experiments were expected to look at the savings 
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factors from the point of view of opportunity costs and effects over 
more than just the period immediately following the actual work. 

The "Qualitative Factors" for each project were summed together 
in a single column. The "Quantitative Factors" for each project were 
also summed. For each project, the "Qualitative Factor" sum was then 
multiplied by the "Quantitative Factor" sum to give a product simulating 
the military worth of that project. This product was then divided by the 
estimated cost of the project in hundreds of dollars to obtain a quotient 
simulating the military worth of the project per unit of project cost. 

The projects were then arranged in order of relative priority with the 
project providing the highest military worth per dollar at the top, and 
the project with the least military worth per dollar at the bottom. This 
priority list was then compared with a priority list of the same projects 
prepared in advance of the experiment by the same evaluator. 

Specific conclusions are found at the end of Appendix I. In 
general, however, the general concept and rationale of the process 
appear sound although some revisions are necessary. A more detailed 
study of the concept, preceded by a comprehensive factor analysis, and 
more extensive experiments at larger installations would be worthwhile. 

III. USE OF THE CONCEPT FOR INTER-ACTIVITY COMPARISON 

Were the system to be used at an activity continuously, it would 
soon be evident that there would develop a certain military worth/dollar 
level above which a project must be rated or it would stand little chance 


of being accomplished. This value might be called the effectiveness 
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level of the activity for it represents the effective return per dollar 
that the activity is able to cain, at the margin, for its maintenance 
effort. If the effectiveness level of the activity i: , for example, 
5.0, then allocation of one more maintenance dollar to the activity 
should result in a military worth return at the 5.53 level. 

[f another activity uses the same system, and utilizes the 
Same criteria for assignment of military wortn factor values, it will 
make similar decisions and also find that it is usually able to accomplish 
those jobs falling above a certain etfectiveness level, but those jobs 
whose rating falls below this effectiveness level never seein to get 
accomplished. This second activity might find that its effectiveness 
levelis 7.0. If this is the case, and ii all factor judgments at both 
activities are made on the basis of the same criteria, then maintenance 
resources should be diverted froin the tirst activity and reallocated to 
the second activity, since 2 more units of military worth per dollar may 
be obtained from maintenance funds at the second activity than at the 
first activity. 

If enough resources are diverted from the first activity, it will 
find that the effectiveness level of its maintenance eftort will rise 
above 5.9. There are now less resources available and only those 
jobs with military worth/dollar ratios of, say, 5.5 or higher seem to 
get accomplished. in the meantime, the second activity, having 
received resources from the first, will find that its effectiveness level 


has decreased, perhaps to 6.5. More resources are now available to do 
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jobs whose military worth/dollar ratio is below 7.0. If this re-allocation 
process continues long enough, the point will be reached where the 
effectiveness levels of the two activities are equal, perhaps at 6.0. 

This would mean that allocation of an additional maintenance dollar to 
either station would result in the same return in military worth. If this 
equilibrium condition existed throughout the Navy at all activities, 

the available maintenance resources would have been distributed in the 
most optimum manner possible, for any reallocation of funds from this 
condition would result in a lowering of the effectiveness level of the 
activity to which the funds are assigned and a raising of the effectiveness 
level of the activity from which the funds were taken. The lower return 
of military worth at one activity would represent an opportunity cost. 

For optimum effectiveness resources should be: re-allocated so that the 
equilibrium condition were regained. 

This concept is analogous to the concepts of marginal consumer 
utility presented in economic theory. The concept illustrates a 
marvelous advantage of being able to measure military worth in terms 
meaningful for inter-activity comparison. The analysis does not say 
whether or not the specific proposed system of approximating a numerical 
value of military worth at each activity would guarantee such splendid 
results for the purposes of resource allocation between activities. 

4For a basic review of marginal utility concepts see: H. H. 


Liebhafsky, The Nature of Price Theory, (Homewood, Ill.: The Dorsey 
Press, Inc., 1963), pp. 82-92. 
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It is probable that a system based on approximations of military 
worth calculated by the activity would not tend itself to automatic 
resource allocation between activities. Each activity would tend to 
warp the system by evaluating their projects tigher In military worth 
than would be the case based on rational evaluation. By so doing, the 
activity would hope to receive additional maintenance resources. At 
the same time, the effectiveness of the system tor allocation decisions 
at the activity might decrease if the system were “warped” unevenly. 

Another problem would be the need for the reviewing agency to 
either treat all activities as equally vital to the needs of the Navy 
(obviously not the case) or else apply priority factors to each of the 
activities to determine proper allucations. If one activity were more 
vital than another, its calculated level of effectiveness could be 
allowed to be lower than that of a lesser activity. The Bureau of Yards 
and Docks should not begin assigning priority values between activities. 
This guidance would have to come from higher authority, who may not 
wish to make this public distinction between activities . 

The conclusion of this study will be that for the near future, a 
system such as proposed in this Chapter would be ineffective in deter- 
mining the proper allocation of maintenance resources between activities. 

IV. SUMMARY 

The concept of a numerical system to approximate an evaluation 
of the military worth of proposed maintenance projects has been proposed 
to assist in allocating maintenance resources at the activity level. The 
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system has been tested in two small experiments as reported in 
Appendix I to this study. The conclusions were that the basic concept 


appears worthy of further investigation on a larger scale. 
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CHAPTER V 
THE MEASUREMENT OF EFFECTIVENESS 

Thus far, this study has discussed what the meaning of effective- 
ness is, the types of programs undertaken by both industry and govern- 
ment in evaluating effectiveness, and one proposal to help in attaining 
increased effectiveness at the local level. This chapter attempts to 
pull together the problems to be encountered in trying to measure main- 
tenance effectiveness, first outlining the characteristics of the "ideal" 
measurement system, followed by a discussion of necessary concepts 
and policies to be followed in determining how the measurement process 
should best be carried out. Recommendations for further study conclude 
the chapter. 

I. THE. IDEALS YSiEM 

The ideal system for measuring effectiveness would have some 
basic characteristics. The result should be of a numerical nature, 
incorporating evaluations of the various important factors and criteria 
contributing to effectiveness. In short, the final results should resemble 
a scorecard, showing scores for the various effectiveness factors, and 
combining these scores in some weighted manner to lead to a logical 
numerical result. The over-all rating should also be in units which 
represent some meaning as to the relative maintenance capabilities of 
the activities evaluated. 

If this precept is accepted, the next questions are: (1) "What are 


the criteria which must be evaluated by the system?", and (2) "What 





are the units in which the over-all effectiveness measure should be 
displayed?" 

Criteria. The actual selection of criteria by which maintenance 
effectiveness is to be judged must come from the top level of the Naval 
organization. Only here can the effects of all various Navy programs be 
felt. If the Secretary of Defense were to establish the criteria for effec- 
tiveness of facilities maintenance in specific, written terms, the measure=- 
ment problem would be cleared considerably, the remaining problem being 
only to decide how best to evaluate the published specific criteria. Any 
complete criteria statement must include a listing of the specific factors 
which affect the capability being measured and the relative weights to be 
assigned to each factor in determining the final composite evaluation. 

One example of defining such criteria is contained in Tables I and II 
of Appendix I. In proposing the use of military worth concepts in Chapter IV, 
the combination of these factors was said to approximate the military worth 
of a particular maintenance project. These criteria undoubtedly need 
revision before Navy-wide acceptance should be proposed, but they at 
least illustrate the type of criteria which must be developed before any 
meaningful measurement attempt can be made. By utilizing published 
criteria of this type, five relatively simple decisions were substituted 
for one more difficult decision, with a reasonable indication that superior 
results may have been achieved. The problem of criteria then resolves 
itself basically to two problems: (1) that of selecting proper factors for 
evaluation, and (2) weighting the selected factors so as to result in a 


rational effectiveness "measurement. ” 
47 





Factors. As aminimum, each of the basic steps toward achieving 
effectiveness should be evaluated. From a review of Chapter II, these 
factors would be: (1) the validity of deficiencies, (2) the ability of the 
organization to correct deficiencies in the right priority order, and (3) 
the efficiency of the activity's maintenance effort for those projects 
accomplished. An evaluation of the effectiveness of the Navy-wide 
maintenance effort would also include an evaluation of the fourth step 
to effectiveness - the over-all ability to allocate Navy maintenance 
resources where most needed. Since this study is primarily concerned 
with evaluation of the maintenance effectiveness within the Navy, this 
factor will be ignored for the moment. It is assumed that if the measure- 
ment system is valid and reliable, then the resource allocation problem 
between activities will be automatically resolved correctly. One purpose 
of the ideal measurement device will be to make this inter-activity 
allocation decision. 

In addition to the three basic factors discussed above, other 
factors need to be evaluated if a more complete picture of the effective- 
ness of an activity’s maintenance effort is to be evaluated. One of these 
factors would be the determination of something akin to the "effectiveness 
level" discussed at the end of Chapter IV. This figure was said to be 
the marginal amount of military worth returned per maintenance dollar 
expended by an activity, and would help suggest where maintenance 


resources should be allocated for maximum benefit. 
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Another factor which may or may not be required, but for which 
provision should be made, would be a rating of relative importance of 
one activity over another. If an allocation decision must be made between 
two activities of otherwise equal maintenance effectiveness, the alloca- 
tion should be made to that activity whose function is of greatest "military 
worth" to the Navy. This possibility has been noted in Chapter IV. The 
Chief of Naval Operations, assisted by the Fleet Activities Command, 
should make this decision when it is required. 

A last major factor which may be helpful would be an over-all 
evaluation of the material condition of the activity resulting from the 
current maintenance effort. This factor would help show how well one 
station is being maintained in comparison with others and would affect 
the degree to which maintenance resources may require readjustment. 

Factor weights. The problem of assigning proper factor weights so 
that the factor evaluations may be combined into one representative 
number is obviously a key one. Nevertheless, there is little new to 
discuss concerning this problem, since the logic is straightforward. If 
one factor is thought to be twice as important as another factor, then the 
value of the first factor should be weighted by a factor of two in comparison 
with the second factor. The only real problem here is the decision as to 
which factors are most important, and by how much in comparison with 
the other factors. This decision requires the same sort of analysis as 
did the actual determination of criteria factors. It is not known whether 


there is any "right" answer to this problem, but the factors should be 
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weighted in some manner to correspond to the professional opinions of 
top management, preferably as supported by some experimental work. 
The only check on the correctness of the factor weights is whether or 
not the resultant combination of factors and weights results in decisions 
which are reasonable to the managers of the programs. This check is 
actually a check of the ability of the system to evaluate both aspects 
of criteria - use of proper factors as well as the proper weights. 
Measurement units. There is no clean answer to the problem of 
proper choice of units for the effectiveness evaluation. Whatever the 
name which may be given to the final unit of measure of effectiveness, 
it should purport to show the relative return of military worth of mainte- 
nance effort per dollar expended at the activity. In addition, the 
separate factors should be scored by some standard numerical technique 
by which that factor could rationally be compared, as an entity, with 
the same factor for another activity. The values of these sub-factors 
have been shown to be subjective in nature, and the units of evaluation 
are really units of “goodness,” or “worth,” or “capability”, which are 
not directly measured from observable physical phenomenon. The units 
involved are rather arbitrary and have abstract meaning, a very undesirable 
situation from a scientific viewpoint. The evaluation attempt, however, 
should serve to improve the maintenance evaluation procedures now in 
use, in much the same way that the fitness report system is intended to 
improve the officer selection process over what would be the situation 


without the evaluation technique. One should not give up the measurement 
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attempt simply because it is not convenient or obvious how the 
measurement should be done. Perhaps the term “evaluation” should 
be substituted for "measurement" when considering problems of the 
type discussed in this study. 

Il. MEASUREMENT OF CRITERIA 

Having outlined the characteristics of the ideal system, the 
next logical step would be to analyze how to go about the actual 
measurement or evaluation of effectiveness at a particular activity. 

It is at this point that the discussion begins to leave the scope of this 
study. It is impossible in this study to comment on specifically how 
each of the effectiveness factors should or could be measured, and no 
experimental work in this area has been conducted within this study. 
Such analyses and experiments should be undertaken as logical extension 
of the concepts outlined in this study. 

There are, however, several basic concepts which should be 
followed in the detailed measurement analysis, when made. 

The first step is to concede frankly the subjective nature of the 
problem. The optimum answer is not just a matter of more detailed 
engineering research and statistical analysis. Engineering or statistical 
studies do serve a worthwhile purpose in those areas where they are 
appropriate (such as the cited studies at the Navy's Civil Engineering 
Laboratory), but subjective analysis is even required in evaluating the 


results of such experiments. 
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A second major consideration is the need for the personnel of the 
Field Engineering Activities to be well informed on the problems and 
needs of the activities within their regions of responsibility. A casual 
once-a~-year walk-through of the activity involved will not be adequate. 
More frequent liaison on the working level must be a part of the measure- 
ment system. In addition, the eventual effect of the Fleet Activities 
Command must be considered and taken into account. The Fleet Activities 
Commander's requirements in an area will be of paramount importance, and 
Field Engineering Office personnel must work closely with this organization, 
being fully cognizant of conflicts which may arise due to the inherent 
conflict of centralized operational authority versus centralized mainte- 
nance support. 

Of equal importance should be the desire of local commanders and 
Public Works Officers to hold the interests of the Navy as a whole ahead 
of the local interests of an activity, where the two may happen to be in 
conflict. Without going into this problem to any great extent, the attempt 
should be made to approach allocation decisions with as nearly an objec- 
tive view as possible, trying not to attach inordinate valuations to local 
projects not in the best interests of the overall broad needs of the Navy. 
Projects which are primarily a result of personal preferences such as 
unnecessary reassignment of office spaces or changes in equipment 
arrangement with no change in capacity should be avoided. One of the 
advantages of the proposed priority system of the previous chapter is 


thought to be the ability of such a system to determine project preference 
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with a minimum of distraction by personal prejudice. The results of the 
analysis cannot be foretold with accuracy by intuitive means. 

The measurement solution may also require assignment of something 
similar to the Level of Maintenance Classification (LMC) Code described 
by the Public Works Management Improvements Draft. : Exception is 
taken in this study, however, to the principle that multiple classifications 
within a particular facility are to be avoided. A hangar may provide more 
direct support to an air station's mission than the administrative offices, 
hence rating a higher maintenance level classification; but the adminis- 
trative office spaces in the hangar may be of less importance to the 
mission of the station than the central administrative offices. Supply 
buildings often provide administrative spaces as well as warehousing 
facilities under one roof. The use of multi-mission facilities necessarily 
requires frequent multi-classifications. Otherwise an illogical system 
is developed which may contribute to more serious allocation errors than 
the original intuitive system. The concept of identifying maintenance 
with the mission of the activity and its tenant activities is certainly 
supported by this study, as has been made clear earlier. 

Another very important analysis and decision to be made is to 


determine the depth to which attempts should be made in factoring, 


dUinited States Department of the Navy, Bureau of Yards and 


Docks, Public Works Management Improvements (Draft), (Washington: 
The Bureau of Yards and Docks, October, 1963), pp. 19-20. 
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weighting, and evaluating various criteria. In all likelihood, the more 
detailed the measurement analysis, the more accurate and meaningful 
the resultant evaluation will be. It will also be more costly than more 
modest efforts. There must exist some level of detailed analysis beyond 
which the added return in knowledge and control does not pay for the 
added expense of evaluation in minutia. Additional control devices and 
information are not desired if they are obtained at inordinate cost. An 
analysis must be made of what the ideal evaluation system is worth to 
the Navy in terms of the additional military worth accruing from better 
allocation of resources, better supervision of field activities, the 
resultant improvement in effectiveness of assigned resources, and the 
ability of the system to substantiate a more effective budget level. To 
this must be compared the cost of obtaining the data to make the system 
work properly. 

The Bureau of Yards and Docks now receives a voluminous amount 
of data having some bearing on the efficiency/effectiveness of the 
facilities maintenance program. Perhaps ninety per cent of the result 
is presently being obtained with only ten per cent of maximum effort. 
Maybe only slightly more (or less) data very similar to that which is 
now available is all that is needed for an optimum system. A review in 
detail of this problem of scope using Operations Research techniques 
should be one of the initial steps in a further look at the measurement 


of facilities effectiveness. 
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The last major concept of advanced measurement analysis 
mentioned here is the need for a teaming of personnel of an Operations 
Analysis or Operations Research background to assist top management 
in resolving the measurement problem. The problem is one that will 
require personnel from many backgrounds if a reasonable solution may be 
expected. No one individual or one approach will suffice for ultimate 
success. 

It may appear unusual that a paper entitled The Measurement of 
Naval Facilities Maintenance Effectiveness has discussed only to a very 
limited extent the actual measurement portion of the problem. It has 
been the stated intent of this study to deal primarily with the conceptual 
aspects of the effectiveness measurement problem, since this basic 
concept appears to be the first need in the solution of the effectiveness 
measurement problem. The conceptual phase of the problem begins to 
fade once the factorial criteria has been defined clearly and the attempts 
are begun to actually perform measurement of these factors. Further dis- 
cussion of specific criteria formats or evaluation techniques must await 
more detailed analysis and experiment. 

III. RECOMMENDATIONS FOR FURTHER STUDY 

During the course of this study it has become evident that much 
more research needs to be done to improve the measurement of maintenance 
effectiveness in the Navy. Some of the more necessary studies are listed 


below. 
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Continue NCEL projects. The recent assignment of statistical 
analysis projects to the Naval Civil Engineering Laboratory represents 
a great stride forward in analysis by the Bureau of Yards and Docks. 
Projects such as those which have been undertaken are just the type 
which may prove of real worth in evaluating the maintenance effort. 

Cost analysis to determine needed measurement depth. This study 
would attempt to establish the proper level of data=collection and control 
beyond which additional data or controls would not be warranted. This 
study would attempt to find out if, indeed, the Navy has perhaps already 
solved "ninety per cent of the effectiveness measurement problem with 
only ten per cent of the effort" such that more detailed information might 
actually not be worthwhile. The results of the study might very well 
indicate that the other studies are impractical at the present time. It 
should be remembered, however, that whenever significant changes 
occur in the parameters of such a study, such as the development of 
inexpensive data-processing equipment, the study must be reviewed, 
for the results may have changed. 

Further experiments with military worth concepts. The attempts to 
develop better means of measuring military worth should be continued, 
not only for allocation of maintenance resources, but for the larger and 
more critical problems in al! sectors of the Navy. Closely aligned with 
this would be further studies to improve the ability to evaluate subjective 


criteria by "quasi-objective” approximations. 
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ocorecard analysis. Experiments with different types of "scorecards 
for activity effectiveness evaluation may be useful in attempting to arrive 
at one combination of various factors which could provide a reasonable 
approximation to the effectiveness evaluation of individual activities. 
Attempts to grade activities, even based on data presently available, 
may result in real advances in use of presently available data for closer 
approximation of maintenance effectiveness for individual activity and 
Navy-wide operations. 

IV. SUMMARY 

This chapter has outlined the characteristics of an ideal measure- 
ment system, discussing the selection of effectiveness criteria and the 
problem of selection of measurement units. Conceptual problems of 
actual measurement are then introduced and recommendations for certain 


further research outlined. 
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CHAPTER VI 
SUMMARY AND CONCLUSIONS 

Implementation of recommendations of the Review of the Manage=- 
ment of the Department of the Navy, Department of Defense policy trends, 
and a probable continuation of present difficulties in obtaining adequate 
facilities maintenance funds will require the Bureau of Yards and Docks 
to develop better means of increasing the effectiveness of facilities 
maintenance. A key aid in increasing effectiveness would be the ability 
to measure the effectiveness of the facilities maintenance program in 
meaningful terms. 

Efficiency is defined as a measure of the technological ability to 
achieve a defined maintenance output with a minimum of resources. 
Effectiveness is concerned with the broader goal of realizing the best 
possible return of maintenance military worth for the resources expended. 
Effectiveness depends on operational efficiency in completing projects 
undertaken, but also depends on optimizing the choice of projects to be 
undertaken, as well as optimizing the allocation of resources among 
different Naval activities to best meet the needs of the Naval Service. 
Through this analysis the need for objective evaluation of subjective 
criteria has been observed. Four basic steps to attain maintenance 
effectiveness are said to be: (1) Determination of valid deficiencies, 
(2) Determination of the relative priority of valid deficiencies, (3) Per- 
forming projects efficiently, and, (4) Proper resource allocation between 


activities. 





A review was made of developments in the maintenance area among 
private industry and public agencies within the State of California. The 
basic industrial concept concerns the use of maintenance resources and 
programs so as to produce, not the lowest cost maintenance, but the 
lowest cost product. The criteria of effectiveness in private industry has 
the same units as the resources involved - money. This criteria cannot 
normally be extended as guidelines for public-supported agencies. 
Although there are some instances of a lack of use of controlled mainte- 
nance techniques by private industry, the advantages of such systems 
appear to be well understood and commonly have been adopted. Much 
sophisticated development work has been done within private industry. 
This work should be continually monitored by the Bureau of Yards and 
Docks. A common feature of maintenance programs of public agencies 
within the State of California was the decentralization of maintenance 
authority and responsibility. From the data available, the acceptance 
of concepts of controlled maintenance appeared to be less within these 
public agencies than within private industry. No clear-cut answer to 
the problem of measurement of maintenance effectiveness within a 
non-profit service organization was found by a review of the literature 
available. 

The study proposes a conceptual system to attempt to aid in 
determining the relative priority order of maintenance work requests at 
the local level. The system attempts to simulate an evaluation of the 


military worth of a proposed maintenance project in cardinal numerical 
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terms, which can be related to the cost of the project to obtain a military 
worth per dollar cost evaluation for each project under consideration. 
Various projects may then be ranked in relative order by placing those 
projects with the highest military wortn per dollar cost valuation at the 
top of the priority list. Two small introductory experiments were held to 
make a rough check of the concept and to indicate whether further study 
of application of the concept would be worthwhile. Conclusions were 
that the results of the use of the system correlated poorly with intuitive 
preference lists, but that the concept should be further evaluated, since 
some evidence was available to indicate that the intuitive preference 
order to which the results of the experiments were compared may not 
have been an appropriate standard for comparison. The intuitive lists 
showed some inconsistencies, further indicating the need for a system 
of a more objective nature to assist in making these decisions. 

An ideal effectiveness measurement system was outlined, providing 
for evaluation of several factors selected to fit effectiveness criteria 
established at the highest Naval level. Suggested factors would be: 

(1) the validity of maintenance deficiencies, (2) the ability of the organi- 
zation to correct deficiencies in the right priority order, (3) the efficiency 
of an activity's maintenance effort, (4) the “effectiveness level" of an 
activity, (5) the relative importance to the Naval mission of one activity 
relative to another (if necessary), and, (6) the material condition of the 
activity being evaluated. Proper factor weights should be selected so 


as to combine separate factor evaluations into the most reasonable 
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composite score. The final units of the measure of effectiveness should 
indicate the relative return of military worth of maintenance effort per 
dollar expended at the activity being evaluated. Several basic concepts 
to be followed in further research to determine actual measurement methods 
for the factors selected were discussed. Further study should involve: 

(1) continuation of current studies at the Naval Civil Engineering Labora- 
tory, (2) cost analysis to determine needed depth of measurement, (3) 
further experiments with military worth concepts, and (4) experiments with 
scorecard analysis to develop a proper evaluation format and measurement 
methods. Of particular importance is the need to decide, on the basis of 
a cost analysis study, the degree to which additional data is needed in 
order to control facilities maintenance programs more effectively than is 


the present caSe. 
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APPENDIX I 
AN INTRODUCTORY EXPERIMENT TO CHECK THE CAFABILITY OF A 
PROPOSED SYSTEM TO ESTABLISH RELATIVE PRIORITY OF 
MAINTENANCE WORK REQUESTS 
I. ABSTRACT 
A proposed system for determining preference order of maintenance 
work requests was examined through introductory experiments performed 
at two small Naval shore installations. Although the preference ranking 
determined by the proposed method did not correlate well with a separate 
ranking of the same work requests by intuitive methods, the calculated 
preference order was deemed acceptable by the activity representatives 
performing the evaluations, with the exception of 2 of the 25 work requests 
in each case. It is believed that the concept tested is rational and 
workable, but that the specific factors and grading system could be 
improved through more detailed study. 
I]. PURPOSE 
The purpose of the experiments was to make an introductory check 
of the concept and ability of a proposed system to develop a workload 
priority listing by determining a simulated military worth per dollar 
cost rating for each of the examined work requests. The results of the 
experiments were to provide background intormation for a proposed use 
of military worth concepts as a portion of a student Research Paper on 
the subject of the measurement of effectiveness of the Navy's Facilities 


Maintenance Program. 
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Ili, EXPERIMENT NO, 1 

Date of Experiment. 16-17 April, 1964 

Location. U.S. Naval Air Facility, Monterey, California 

Procedure. Only work requests estimated to cost more than $100 
were utilized in the experiment in order to stay out of the range of service 
call project scope. A sample size of 25 work requests were arbitrarily 
picked as being large erough to make a preference ranking moderately 
difficult by normal intuitive means and yet not so large as to make 
calculation of the derived ranking order too time-consuming, considering 
this as only an introductory experiment. 

In cooperation with the Public Works Officer at NAF, Monterey, 
twenty~<-five typical work requests were selected for review. An attempt 
was made to include a cross-section of typical work for the activity. The 
current backlog was not large enough to supply the number of work requests 
needed, so some recently completed work requests were also used. In 
the case of the completed work, the estimated cost rather than the com- 
pleted cost figures were used. The Public Works Officer agreed to act 
as evaluator for the experiment. 

The twenty-five work requests were listed in random order and 
described for future reference on the identification sheet (Table III). 

The evaluator then arranged these work requests in order of desirability 
by intuitive process, attempting to pick the order which would result 
in the maximum military worth per dollar expended. local concern for 


any activity “pet projects” was to be ignored. 
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After the work requests had been arranged in intuitive preference 
order, the ranking order was noted in the last column of the identification 
sheet (Table III) for future comparison with the eventual calculated order. 
The work requests were then replaced into their original random order and 
the evaluator was asked to assign numerical grades for each of the five 
qualitative and qualitative factors as described by Tables I and I]. The 
results of his grading decisions were placed in columns 2, 3, 5, 6 and 7 
of the data sheet, as shown in Table IV. 

For each work request, the two qualitative factor grades were added, 
with their sum appearing in column 4 of the data sheet. For each work 
request, the three quantitative factors were added, the Sum appearing 
in column 8 of the data sheet. For each work request the sum appearing 
in column 4 and column 8 of the data sheet were multiplied together, 
the result appearing in column 9. The figure in column 9 was then 
divided by the estimated work request cost in hundreds of dollars and 
the result placed in column 10. The calculated ranking order was then 
determined, with the work request having the largest number in column 10 
placed as first priority, and the work request with the smallest number 
in column 10 placed as the iast priority and least desirable of the list. 
The conceptual background for these calculations appears in chapter IV 
of the body bi this Research Paper. The five factors and the descriptive 
comments uSed as a guide in their evaluation are shown in Tables I and 
II. These factors were determined by intuitive analysis and are not 


based on other experimental analysis or data. 
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The calculated preference order was then noted in column 1! of 
the data sheet for comparison with the original intuitive order shown 
in the last column of the identification sheet. Aiter reviewing the pre- 
liminary results, the evaluator decided to review his factor evaluations, 
particularly with respect to the absolute savings column of the quanti- 
tative factors. A new data sheet wes prepared showing all sums and 
the changed factor scores (Table V). The preference calculations were 
re-calculated in the Same manner as described previously and the second 
calculated preference order was shown in column Il of Table V. A graph 
was made plotting the original intuitive preference order of each work 
request against its calculated preference order (Figure 1.). 

The work requests were then iaid out on a4 table in order of the 
final calculated preference order, and the resuits of the experiment 
discussed. 

Discussion. A review of the first results indicated that there may 
have been some misunderstanding as to the definition and meaning of 
the quantitative cost factor. Scores for this factor were to have been 
based on absolute values of cost savings, rather than in comparison 
with the cost of the project. Some of the factor grades were subsequently 
revised on the data sheet of Table V, leading to a revised preference 
order. This second order is considered the result of this experiment, 
and shall be the only order discussed here. 

Figure 1 shows that there was very little, if any, correlation 


between the first intuitive preference order and the final calculated 
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order. This means that either the system used was ineffective or that 
the intuitive order was questionable. 

In the review of the calculated preference order, the evaluator 
stated that there were only two projects that he desired to shift by a 
significant amount. He felt that work request 5017 ((D# 16) which was 
tenth in the calculated preference order should be placed twentieth or 
lower, After discussion it appeared that the real problem here was one 
of deficiency validity. The activity administration apparently considered 
the deficiency a valid one, although the evaluator disagreed. Given 
the basic assumption that the deficiency was valid, its low cost would 
tend to place the job in the top half of the list. The quantitative 
mission factor is relatively high since all men entering the enlisted 
mess would enter through the proposed door. 

The other project for which it was tfeit a shift should be made was 
work request number 3657 ¢D# 2) for installation of transformers. The 
evaluator felt that the job should be placed sSigniticantly higher than 
the twenty-second priority calculated for it. On questioning, however, 
it was determined that: (1) the power deficiency for the area involved 
was not critical, (2) the job had been proposed for some time, but had 
never started since personnel were needed for more critical projects, 
and (3) the cost is very high in comparison with the other projects in 
the sample. 

The over-all comments on the calculated preference order were 


that the list seemed to have the most critical repair jobs towards the 


06 





top, followed next by less critical jobs of an appearance nature, such 

as painting, etc., then followed by alteration and improvement work 

of lesser importance from a maintenance point of view. The evaluator 
felt that the approach taken was a rational one, and that, with the two 
questionable exceptions mentioned above, the list generated was very 
workable. The calculated preference order was thought to be just as 

good as the original list, if not better. The evaluator was very interested 
in the technique and results of the experiment, although the size of his 
organization and workload is so small that a formal decision system may 


be of little real assistance. 
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TABLE I 


QUALITATIVE FACTORS 


RANGE MISSION ESSENTIALITY 
Contributes little or nothing 
1~20 to ability of station to meet 


its mission requirements. 
Not necessary for support 
elements. Desirable to 

maintain or improve aes= 


OPPORTUNITY COST SAVINGS 
Project will interfere with 
normal maintenance. Cost of 
project will not be returned 
by future savings. 


thetic Or appearance. 


Helpful in meeting operational 
21-40 requirements. Desirable to 

enable supporting elements 

to meet operational needs. 

Important to morale and main- 

tenance of appearance of 


acthvity. 
Desirable for easing of op- 
41-60 erational problems. Impor- 


tant to support elements to 
help meet operational re- 
quirements. Important to 
morale of base personnel 


Project will eventually 
realize return of the invest- 
ment involved. Necessary to 
prevent a needed repair from 
degenerating to one of larger 
scope. 


Project would realize signi-= 
ficant savings in the first 
year after completion. Would 
avoid major change in main-~ 
tenance or repair scope by 
present accomplishment. 


and appearance of activity. 


Important to fulfillment of 
61-80 Operational needs. Drastic 
substitutes needed other- 
wise. Essential to proper 
support of operational 
needs. A most critical 
morale or appearance re= 


Project would save up to 
twice its cost within the 
first year after completion. 
Necessary to avoid large 
repair work which would 
otherwise he required. 


quirement. 


Directly and completely ap- 

81-100 plicable to essential opera- 
tional needs of the activity. 
Critical to enable support- 
ing elements to meet 
requirements of necessary 
operations. 


Project would Save more than 
twice its cost within the first 
year after completion. Neces- 
sary to avoid complete re- 
placement of the facility or 
grossly excessive repair 
which would otherwise 


develop. 
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ID# 
lg 
2 
3 
4 
o 
6 
7 
8 
3 


OE: 
A 
Lae 
13. 


ia. 
PD. 


16. 
i? 
18. 
no 
a. 
ZA, 


Cees 
Wie 


24. 


ZS 


W.O.# 


4007 
3667 


4079 
4060 


4008 
5024 


3716 


3989 


3964 


9031 


59034 


5030 


4031 


4100 
5015 


5017 
5018 
5022 
5028 
9038 
Sus 


5040 
9041 


5042 


5044 


TABLE III 


IDENTIFICATION SHEET (Experiment #1) 


Description 
Minor repair & paint rear bldg l 


Install transformers boiler room 
bldg 6 

Lay insulation in overhead bldg 1 

Replace wooden steps w/concrete, 
bldg. l 

Curb & Gutter N. side bldg 53 

Repairs to asphalt A/C parking 
apron 

Install fluorescent fixtures, 
hangar 101 

Repaint trim, windows & doors, 
hangar 101 

Re-install exhaust system in 
welding shop 

Remove wall, renovate hopper 
room 

Rehab. I & E lecture room, 
bldg. 15 

Install fluorescent fixtures 
bldg. 37 

Patch/repaint interior hobby shop 
garage 

Paint overhead, galley dining room 

Install safety valve piping through 
roof 

Install door at West side of mess 
hall 

Misc. repair/repaint, galley 
kitchen 

Maint. to hangar 6 

Maint. to street signs and markings 

Install wall paper in galley dining 
room 

Portable partitions for galley 
entrance 

Replace EM parking area headers 

Maint. to fences, parking lots & 
Signs : 

Repair ceramic tile, galley butcher 
shop 

Repair floor behind Acey~Ducey bar 


V2 


5 Cost 


419 


3808 
1010 


199 
606 


1505 


1454 


701 


157 


ZG 


1496 


222 


480 
160 


Zile 
ms 
837 
610 
395 
341 


465 
L45 


374 
107 


Inti. 
Order 
7 


Zo 
10 


i 


20 
[> 
16 


Li 
18 


Ne, 


ae 
24 


14 





TABLE IV 
DATA SHEET (Experiment #1 - preliminary) 





QUATIIATIVE QUANTITATIVE 
] 2 3 4 5 6 7 8 9 10 ll 
# Miss. Savng. Sum Miss. Savng. Urg. Sum Prod. Prod/Cost Order 
im 30 25 So so aS -4 1.2 466.0 15.8 7 
a oo 25 80 6 26 oa 1.4 See 2.9 22 
or. 10 a5 35 od a ol o> (in. ia 7 25 
mm 25 oo 60 oa ae) 4 1.1 66.0 Bonz 4 
sp. 45 20 65 wd a a: 6 en0 6.4 16 
6. 15 15 30 ao aS od 9 27.0 1.8 24 
7. £45 50 100 so 6 2 2.3 30.9 § .9 10 
So. ES ZS 40 wo a35 3 9 “S56 .0 aL 19 
oe, 45 eS 60 4 a oe  L.2 72 oo 45.9 2 
10 Zo 15 40 nee. ne Re 6 24.0 8.5 LZ 
rt 45 Zo 70 4 sa3 a2 9 "63.0 4.2 eal 
EZ 25 a5 50 6 4 <2 74.2 VOR 27.1 6 
13 30 ZS 55 ao aS od. of “eg. 10.3 8 
14 35 25 60 24 oe ae oo 8.0 30.0 5 
15 Zo 15 40 4 ea ol 6 24.0 8.7 la 
16 10 10 20 | a! Pei 6 12.0 Oe 9 
17 35 2'5 60 ae a, 4 8 48.0 5.7 18 
18 30 25 55 <3 2 so 8 44.0 Jee 14 
19 20 Z5 45 me sik ou 4 “e.0 Sa 20 
20 30 15 50 ot ol od 4 29.0 S40 Ae 
Je 29 5 30 | Pek sa 3 o20 1.9 23 
22 15 Zo 40 au ae | ou od 212-0 8.3 13 
23 20 21 4] od ot met 4 16.4 Tel 15 
24 40 40 80 pee ot ./ 0.6 28.0 34.2 3 
ZS 39 40 vO we of 2-6 91.5 [kz 75 405.0 l 


73 








are 
oP oe ORE EEE— Crees 





ie 
$¢+4@ 6 GOOG See 


TABLE V 
DATA SHEET (Experiment #1 - Revised) 





QUALITATIVE QUANTITATIVE 
] Z 3 4 5 6 : 8 9 10 11 
. Sum Miss. Savng. Urg. .  Prod/Cost Order 
i 50 bo a 4 1.0 a8 Looe if 
2. 80 .6 4 2 Le S36 2e5 Ze 
Ss . 35 3 1725 1.7 24 
4. 60 wl o4 4 9g 54 27 A 6 
2 65 a2 ev a2 S Sao 5.4 17 
S. 30 eo el ail st Zt 1.4 25 
7 100 so 8 ue Les 150 VOeS 8 
8. 40 9 36 5a) 19 
9. 60 4 1 8) 1.1 66 42.0 2 
10 40 Z 1 2 2 20 Tie 1S 
ll 70 9 63 A.@ el 
12 50 6 5 2 Ls 65 293 5 
is 39 J 49.5 Oe 9 
14 60 8 48 30.0 4 
15 40 6 24 S.7 aPZ 
16. 20 6 12 id.l1 10 
te? . oo Bis 70 ez .6 ~4 i.2 84 LO. 1 
18 So 8 44 Teed 14 
19. 45 4 18 Sat 20 
20. 50 4 20 oo 16 
Za. 30 Ss g 1.9 23 
Ze. 40 3 12 8.3 13 
23 4 ] 1 l 3 12.3 5.4 18 
24 80 1.6 128 S4e2 3 
25 15 Pes Li225 “rose 0 1 
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FIGURE 1 


PLOT OF CALCULATED PREFERENCE ORDER AGAINST 
INTUITIVE PREFERENCE ORDER (EXPERIMENT #1) 
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IV. EXPERIMENT NO .-2 

Date of Experiment. ¢i=-22 April, 1964 

Location. U.S. Naval Postgraduate School, Monterey, California 

Procedure. The procedure used in experiment no. 2 was identical 
with that of experiment no. i except that the graded factor evaluations 
were not revised after the first results. Only one calculated preference 
order was developed. The identification sheet for this experiment is 
table VI. The data sheet for this experiment is table VII. The graph of 
original intuitive order against calculated order is figure 2. In this 
experiment the Public Works Officer did not perform as evaluator although 
he had been consulted prior to carrying the experiment out. He designated 
his Director of Maintenance Control to perform the experiment. This indi- 
vidual is the one, within this organization, who normally makes the 
basic priority and scheduling deciSions. 

Discussion. Figure 2 shows that the calculated preference order for 
the second experiment has more correlation with the original intuitive order 
than was the case with the first experiment. Comments of the experi- 
menter, however, were not as constructive, although they are worthy of 
note. 

Again there were two jobs that were said to need adjustment from 
the preference order calculated for them. The first of these was work 
request 2871, (ID# 25) for work in a chemistry laboratory at an estimated 
cost of $525. It was stated that the Public Works Department would 


be administratively directed to do this work prior to the time it would be 
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accomplished at the 14th priority level. In view of this fact the 
evaluation grade for urgency (column 7) should have been higher than 
.3, and, if the requirement were a valid one, the costs of renting or 
leasing the required laboratory facilities if the work were not accom- 
plished (absolute cost savings ~ column 6) should also have been scored 
higher than .3. Assignment of .7 evaluations to both of these factors 
would only bring the project from 14th to 9th, however, due to the 
medium cost of the project. 

The other job requiring adjustment was work request 2858 (ID# 19), 
a large miscellaneous repair project. It was felt that this job should be 
moved up the list a significant amount, as a very worthwhile project. 
It is interesting to compare this job with work request 2859, a much 
smaller job of the same nature in the same building. The smaller job had 
a 20% higher qualitative savings score than the large one, but a quantitative 
savings score of more than double the large one. This result is an apparent 
error on the part of the evaluator. It should be noted, however, that 
assignment of maximum value of 1.0 to the quantitative savings factor 
(column 6) would only result in placing the job in twentieth position vice 
the current twenty~second position. 

It was noted that project costs varied from a minimum of $120 to 
a maximum of $4407 = 36 times the minimum cost project. If a minimum 
value of .1 is assigned to each quantitative factor, the possible range 
of the quantitative factors sum will be from .3 to 3.0 = 10 times the 


smallest score. The actual range of the quantitative sums was from a 
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minimum of .3 to a maximum of 2.1 - 7 times the minimum score. 
Although the largest project had more than 36 times the cost of the 
smallest, quantitative factors could only have varied by a factor of 10, 
and actually did vary only by a factor of 7. As the system is now con- 
stituted it would be very difficult to place an expensive project near the 
top of the list, regardless of its true value in relation to the others. A 
modification is needed for the quantitative scale to at least provide for 
the same military worth range as the range of costs for the projects 
being considered. This might be done by utilizing open-ended quanti- 
tative scales which would have no upper value ceiling. Average values 
could be indicated for projects of normal scope, but provision for large 
variations from the norm could be made. A detailed study should be able 
to combat this problem in some manner that would not increase the com- 
plexity of the method. 

The evaluator started to comment that work request 2833 (ID# 15), 
new electrical service for the picnic grounds, should be moved up from 
the calculated order position (ninth) due to the need to straighten out the 
present "jury rig" power service. It was then noted, that the original 
intuitive list had shown the project 16th, and the project priority had 
already been raised from that position by the calculations. 

Another comment was that work request 2841 (ID# 10) shouldn't 
be placed as high on the list as was calculated (sixth). During another 
portion of the discussion, however, it had been pointed out that this 


project to replace natural gas service was not an operational need, but 
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a safety hazard which could potentially involve a major part of a 
large facility. It would be inappropriate to lower this priority ficure 


unless the hazard is of a minor nature. 
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a. 2675 
4, 2680 
De 273 
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5. 2784 
oI. 2997 

eo Z799 

ue 2800 

2 2807 

Ss 2874 

14, 2828 

iS: 2833 

Ore 2841 

17. 2848 

ne. 2857 
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Zl, 7366 

Bide. 2827 
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TABLE VI 
IDENTIFICATION SHEET (Experiment #2) 


Description _. $ Cost 

Attic lights, crawl beds, bldg 301 446 
Furred overhead & lighting bldg 

229 1136 
Exhaust fan in disbursing office Zee 
Maintenance to steam manholes 192 
Replace tile various rooms bldg 232 121 
Partitions room 105, bldg 232 642 
Maintenance to bldg 232 650 
North & East banks erosion control 477 
Repair wall in hall 094, bldg 220A 196 
Maintenance to bldg 234 329 
3 test lead racks, bldg 232 125 
4 signs for La Mesa Village 300 
Rehab. bathroom, room 3ll, 

bldg 322 500 
Remove meat tracks, bldg 220 326 
New electrical service, picnic grnds. 248 
Repipe natural gas service ,Aerolab oid 
Repaint rooms, 406, 309, bldg 221 25 2 
Maintenance to bldg 322 783 
Maintenance to bldg 322 4407 
Maintenance to bldg 322 168 
Subterranean drainage, 122 Morrell 440 
Maintenance to bldgs 213 thru 217 ZOD 
Relocate slide rear of J-115 120 
Maintenance to J bldgs 330 
Tables, sink, water, etc., chem. 

lab. 525 
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TABLE VII 
DATA SHEET (Experiment +2) 


QUALITATIVE QUANTITATIVE 
1 2 3 4 5 6 7 8 9 10 li 
Prod. Prod/Cost Order 





Ls 9 7 12 1 ae au 4 4.38 Wael ae 
2. 40 25 65 ay: a2 a2 <6 .6ae30 3.4 21 
3. (7238 18 46 az ae one 0 2060 owe 15 
4, 90 60 1350 08 of 6 2.1 315.0 164.0 1 
5. 18 ZS 43 ae eZ Bi oo gue ozilleee 7 
6. 30 Ze oe oe | a2 oJ 36.4 Sof 18 
7. 69 24 89 4 20 6 1.5 UGa%o era 8 
as. 0 4. 54 a of one L.38 #@we. aa? 10 
a S 21 26 od ol ae. 4 10.4 Jo 19 
10 63 22 85 ae a, 20 1.2 Wao “3120 S. 
ll 45 ao 70 a Se o4 ./ 4949 "3972 4 
eZ 45 15 60 4 aa oa o&  doe0 | Zed YZ 
13 40 50 90 ol he of 1.6 144.0 72.0 3 
14 10 9 IS oe! eZ ol =) ies Zs 24 
1S o 30 35 Be av ae bel “38.5 Lome 9 
16 21 26 47 a AS) 6 1.4 /050 "2250 6 
im, § ls 5 20 oul ou 2 4 8.0 SS 20 
18 60 30 90 we ao o4 lot 3.0 Aa? il 
19 60 50 110 Bee - ay, 1.3 143.0 3.3 ae 
20 60 60 120 me JI a 1.8 216.0 128.5 2 
ZA 30 20 90 mae we 3 -6 30.0 6.8 Le 
Zz 20 15 35 eZ ou 2 oD | eyo £38 16 
23 9 5 10 ad ol ot a0 3.0 Zins Zo 
24. 22 Zo 47 BZ 04 1 of oad Dae 13 
25. 46 eZ 68 se 3 x 7 47.6 ost 14 
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PLOT OF CALCULATED PREFERENCE ORDER AGAINST 
INTUITIVE PREFERENCE ORDER (EXPERIMENT #2) 
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V. CONCLUSIONS: BOTH EXPERIMENTS 

The experiments did net, in fact, clearly determine whether or 
not the military worth system proposed here was the proper system for 
establishing workload priority. The calculated preference order in both 
cases correlated very poorly with the preference order established by 
intuitive means. This fact alone, however, does not tel! one that the 
system is no good. There is considerable evidence that the intuitive 
method may have developed several failings in determining the order in 
which maximum worth would be retumed to the Navy. 

There was no strong opposition to either of the calculated priority 
lists developed, even though such poor correlation developed between 
the calculated and intuitive lists. This would lead one to wonder if, 
with the exception of projects at the very high and low ends of the 
scale, intuitive means of decision in such a problem are incapable of 
determining the relative merits of projects in the mid-range of deSsira-= 
bility. Perhaps local pressures play a greater part in decision-making 
than logical and rational analysis based on the over-all goals of the 
Navy. Intuitive decisions on the transformer project in experiment I, 
and the picnic ground and natural gas service of experiment 2 are at 
least questionable. 

It has already been noted in the discussion of experiment no. 2 
that the present scale structure does not provide opportunity of adequately 
considering the possible benefits of projects with very large scope. The 


sum of quantitative factors would not uSually vary by more than a factor 


G3 





of 10, while the magnitude of scope measured in cost figures varies 

in both instances by factors greater than 30. In both cases, one of 

the two projects whose calculated ranking was considered incorrect 
involved jobs of very high cost which had been ranked near the bottom 
of the preference scale. It is recommended that future studies of a 
similar nature be structured so as to be able to give large projects 
proper cost effectiveness factor ratings when they are deserved. A 
future experimenter would be cautioned, however, not to co to extremes 
in obtaining benefits for large jobs. The larger projects are exactly 
the ones which should receive the greatest test for military worth, 
simply due to the fact that a major investment is involved. They should 
be made to prove their value in competition with smaller projects. It 
should not be possible, however, to improve a large job’s ranking 
merely by substituting for it two projects of half the scope, such as 
might be the case under the present system. 

It should also be noted that any system of this nature which tends 
to “impersonalize" priority determination will be useless if the command 
attitude at the activity is not receptive to the idea. It must also be 
recognized that a Commanding Officer, for various reasons, will feel 
that he must occasionally contravene such a system in carrying out his 
responsibilities. This does not say the system is not worthwhile. If 
the system does nothing more than assign relative desirability to stan=- 
dard work requests, being modified for particular non~standard reasons, 


it will have performed a fine "management=-by-exception” function. It 
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is certainly not suggested here to do away with higher-level review of 
workload assignment just because of a new "Scientific" system. 

A system such as this should be helpful in structuring the deci- 
sion thought process. When there are problems in determing which of 
two jobs are more important, a look at the factor scoring sheet will 
help indicate why one job should be preferred over another. The system 
has the effect of putting the thought process on paper for all to observe. 
The reasons for a decision are more obvious and subject to more rational 
analysis. This feature of the system was frequently observed when 
discussing why one project varied greatly in preference order from its 
original intuitive position. 

The comment must be made that there is no known objective way 
of measuring the results of this experiment. This would involve measuring 
the value of the intuitive system against the value of the system of calcu= 
lations and evaluations. If such a measurement system is available, then 
this project should be abandoned and the measurement device used to 
determine whether the Navy is doing the right maintenance job or not. 
The only means of evaluating such an experiment at the present time is 
through a subjective analysis of the results by experienced maintenance 
personnel with the results of the analysis expressed as professional 
opinion. Both evaluators in the experiments expressed interest in the 
system as used, and confidence in the basic logic through which the 
system was developed. The author also feeis that the concept is 


logically sound and that detailed work in this area could arrive at the 





structure of a similar system which would be easily applied at the local 
level. These are the “professional” opinions obtained in this case. 


Opinions by those with superior background would be desirable. 























